Switching hearing implant coding strategies

ABSTRACT

An input sound signal is processed to generate band pass signals. Band pass envelope signals are extracted that reflect time varying amplitude of the band pass signals. Stimulation timing signals are generated that reflect the temporal fine structure features of the band pass signals. A key feature value present in the input sound signal or the band pass signals is monitored. For one or more selected band pass signals, a stimulation coding strategy is selected from multiple possible stimulation coding strategies based on the key feature value. The multiple possible stimulation coding strategies including an envelope-based stimulation coding strategy based on the band pass envelope signals, and an event-based stimulation coding strategy based on the stimulation timing signals. The selected stimulation coding strategy is used to produce the electrode stimulation signals for the electrode contacts. The arrangement automatically switches between different selected stimulation coding strategies as a function of changes in the key feature value.

This application claims priority from U.S. Provisional PatentApplication 62/174,003, filed Jun. 11, 2015, and from U.S. ProvisionalPatent Application 62/215,187, filed Sep. 8, 2015, both of which areincorporated herein by reference in their entireties.

TECHNICAL FIELD

The present invention relates to signal processing arrangements forhearing implants, and more particularly, to automatically transitioningspeech coding strategies for cochlear implants.

BACKGROUND ART

As shown in FIG. 1, sounds are transmitted by a human ear from the outerear 101 to the tympanic membrane (eardrum) 102, which moves the bones ofthe middle ear 103 (malleus, incus, and stapes) that vibrate the ovalwindow and round window openings of the cochlea 104. The cochlea 104 isa long fluid-filled duct wound spirally about its axis for approximatelytwo and a half turns. It includes an upper channel known as the scalavestibuli and a lower channel known as the scala tympani, which areconnected by the cochlear duct. The cochlea 104 forms an uprightspiraling cone with a center called the modiolus where the spiralganglion cells of the acoustic nerve 113 reside. In response to receivedsounds transmitted by the middle ear 103, the cochlea 104 functions as atransducer to generate electric pulses which are transmitted to thecochlear nerve 113, and ultimately to the brain which perceives theneural signals as sound.

Hearing is impaired when there are problems in the ability to transduceexternal sounds into meaningful action potentials along the neuralsubstrate of the cochlea 104. To improve impaired hearing, auditoryprostheses have been developed. In some cases, hearing impairment can beaddressed by a cochlear implant (CI), a brainstem-, midbrain- orcortical implant that electrically stimulates auditory nerve tissue withsmall currents delivered by multiple electrode contacts distributedalong an implant electrode. For cochlear implants, the electrode arrayis inserted into the cochlea 104. For brain-stem, midbrain and corticalimplants, the electrode array is located in the auditory brainstem,midbrain or cortex, respectively.

FIG. 1 shows some components of a typical cochlear implant system wherean external microphone provides an audio signal input to an externalsignal processor 111 which implements one of various known signalprocessing schemes. For example, signal processing approaches that arewell-known in the field of cochlear implants include continuousinterleaved sampling (CIS) digital signal processing, channel specificsampling sequences (CSSS) digital signal processing, spectral peak(SPEAK) digital signal processing, fine structure processing (FSP) andcompressed analog (CA) signal processing.

The processed signal is converted by the external signal processor 111into a digital data format, such as a sequence of data frames, fortransmission by an external coil 107 into a receiving stimulatorprocessor 108. Besides extracting the audio information, the receiverprocessor in the stimulator processor 108 may perform additional signalprocessing such as error correction, pulse formation, etc., and producesa stimulation pattern (based on the extracted audio information) that issent through electrode lead 109 to an implanted electrode array 110.Typically, the electrode array 110 includes multiple stimulationcontacts 112 on its surface that provide selective electricalstimulation of the cochlea 104.

An audio signal, such as speech or music, can be processed into multiplefrequency band pass signals, each having a signal envelope and fine timestructure within the envelope. One common speech coding strategy is theso called “continuous-interleaved-sampling strategy” (CIS), as describedby Wilson B. S., Finley C. C., Lawson D. T., Wolford R. D., Eddington D.K., Rabinowitz W. M., “Better speech recognition with cochlearimplants,” Nature, vol. 352, 236-238 (July 1991), which is herebyincorporated herein by reference. The CIS speech coding strategy samplesthe signal envelopes at predetermined time intervals, providing aremarkable level of speech understanding merely by coding the signalenvelope of the speech signal. This can be explained, in part, by thefact that auditory neurons phase lock to amplitude modulated electricalpulse trains (see, for example, Middlebrooks, J. C., “Auditory CortexPhase Locking to Amplitude-Modulated Cochlear Implant Pulse Trains,” JNeurophysiol, 100(1), p. 76-912008, 2008 July, which is herebyincorporated herein by reference).

However, for normal hearing subjects, both signal cues, the envelope andthe final time structure, are important for localization and speechunderstanding in noise and reverberant conditions (Zeng, Fan-Gang, etal. “Auditory perception with slowly-varying amplitude and frequencymodulations.” Auditory Signal Processing. Springer New York, 2005.282-290; Drennan, Ward R., et al. “Effects of temporal fine structure onthe lateralization of speech and on speech understanding in noise.”Journal of the Association for Research in Otolaryngology 8.3 (2007):373-383; and Hopkins, Kathryn, and Brian Moore. “The contribution oftemporal fine structure information to the intelligibility of speech innoise.” The Journal of the Acoustical Society of America 123.5 (2008):3710-3710; and all of which are hereby incorporated herein by referencein their entireties).

Older speech coding strategies mainly encode the slowly varying signalenvelope information and do not transmit the fine time structure of asignal. More recent coding strategies, for example, Fine StructureProcessing (FSP), also transmit the fine time structure information. InFSP, the fine time structure of low frequency channels is transmittedthrough Channel Specific Sampling Sequences (CSSS) that start atnegative to positive zero crossings of the respective band pass filteroutput (see U.S. Pat. No. 6,594,525, which is incorporated herein byreference). The basic idea of FSP is to apply a stimulation pattern,where a particular relationship to the center frequencies of the filterchannels is preserved, i.e., the center frequencies are represented inthe temporal waveforms of the stimulation patterns, and are not fullyremoved, as is done in CIS. Each stimulation channel is associated witha particular CSSS, which is a sequence of ultra-high-rate biphasicpulses (typically 5-10 kpps). Each CSSS has a distinct length (number ofpulses) and distinct amplitude distribution. The length of a CSSS may bederived, for example, from the center frequency of the associated bandpass filter. A CSSS associated with a lower filter channel is longerthan a CSSS associated with a higher filter channel. For example, it maybe one half of the period of the center frequency. The amplitudedistribution may be adjusted to patient specific requirements.

For illustration, FIG. 2A-2B show two examples of CSSS for a 6-channelsystem. In FIG. 2A, the CSSS's are derived by sampling one half of aperiod of a sinusoid whose frequency is equal to the center frequency ofthe band pass filter (center frequencies at 440 Hz, 696 Hz, 1103 Hz,1745 Hz, 2762 Hz, and 4372 Hz). Sampling is achieved by means ofbiphasic pulses at a rate of 10 kpps and a phase duration of 25 μs. ForChannels 5 and 6, one half of a period of the center frequencies is tooshort to give space for more than one stimulation pulse, i.e., the“sequences” consist of only one pulse, respectively. Other amplitudedistributions may be utilized. For example, in FIG. 2B, the sequencesare derived by sampling one quarter of a sinusoid with a frequency,which is half the center frequency of the band pass filters. TheseCSSS's have about the same durations as the CSSS's in FIG. 2A,respectively, but the amplitude distribution is monotonicallyincreasing. Such monotonic distributions might be advantageous, becauseeach pulse of the sequence can theoretically stimulate neurons at siteswhich cannot be reached by its predecessors.

FIG. 3 illustrates a typical signal processing implementation of the FSPcoding strategy. A Preprocessor Filter Bank 301 processes an input soundsignal to generate band pass signals that each represent a band passchannel defined by an associated band of audio frequencies. The outputof the Preprocessor Filter Bank 301 goes to an Envelope Detector 302that extracts band pass envelope signals reflecting time varyingamplitude of the band pass signals which includes unresolved harmonicsand are modulated with the difference tones of the harmonics, mainly thefundamental frequency F0, and to a Stimulation Timing Module 303 thatgenerates stimulation timing signals reflecting the temporal finestructure features of the band pass signals. For FSP, the StimulationTiming Module 303 detects the negative to positive zero crossings ofeach band pass signal and in response starts a CSSS as a stimulationtiming signal. A Pulse Generator 304 uses the band pass envelope signalsand the stimulation timing signals to produce the electrode stimulationsignals for the electrode contacts in the implant 305.

FSP and FS4 are the sole commercially available coding strategies thatcode the temporal fine structure information. Although they have beshown to perform significantly better than e.g. CIS in many hearingsituations, there are some other hearing situations in which nosignificant benefit has been found so far over CIS-like envelope-onlycoding strategies, in particular with regard to localization and speechunderstanding in noisy and reverberant conditions.

Temporal fine structure might be more affected by noise than theenvelope is. It might be beneficial to use fine structure stimulationdepending, for example, on the signal of noise ratio or on the dynamicreverberation ratio. In existing coding strategies, the use of thetemporal fine structure is adapted in a post-surgical fitting sessionand is not adaptive to the signal to noise ratio.

SUMMARY

Embodiments of the present invention are directed to systems and methodsfor generating electrode stimulation signals for the electrode contactsin a cochlear implant electrode array. A preprocessor filter bank isconfigured to process an input sound signal to generate band passsignals that each represent a band pass channel defined by an associatedband of audio frequencies. An envelope detector is configured to extractband pass envelope signals reflecting time varying amplitude of the bandpass signals. A stimulation timing module is configured to generatestimulation timing signals reflecting the temporal fine structurefeatures of the band pass signals. A key feature monitor module isconfigured to monitor a key feature value present in the input soundsignal or the band pass signals. A stimulation coding module isconfigured to: i. select a stimulation coding strategy for one or moreselected band pass signals from a plurality of stimulation codingstrategies based on the key feature value, wherein the plurality ofstimulation coding strategies include: (1) an envelope-based stimulationcoding strategy based on the band pass envelope signals, and (2) anevent-based stimulation coding strategy based on the stimulation timingsignals, and ii. switch between different selected stimulation codingstrategies as a function of changes in the key feature value. Thestimulation coding module is configured to autonomously select theappropriate coding strategy at each point in time. A pulse generator isconfigured to use the selected stimulation coding strategy to producethe electrode stimulation signals for the electrode contacts.

In specific embodiments, the stimulation coding module may be configuredto switch between different selected stimulation coding strategiesdirectly without a transition period of time, or it may switch betweendifferent selected stimulation coding strategies by adaptively changingthe selected stimulation coding strategy over a transition period oftime to become a different stimulation coding strategy; for example,either after or while the key feature value changes from an initialvalue to a coding change value. One of the stimulation coding strategiesmay use adaptive stimulation pulse rates and another stimulation codingstrategy may use constant stimulation rates.

The key feature value may be a signal to noise ratio (SNR) or a directto reverberation ratio (DRR) of the input sound signal; or envelopeslope, envelope peak, or envelope amplitude of the band pass envelopesignals. The stimulation coding module may be configured to select andswitch between coding strategies for a single band pass signal, or formultiple selected band pass signals—e.g., using an n of m strategy orfor a dominant channel having a largest key feature value and aplurality of adjacent neighboring channels.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows anatomical structures of a human ear and some components ofa typical cochlear implant system.

FIGS. 2A and 2B show channel specific sampling sequences (CSSS) for two6-channel systems utilizing biphasic pulses at 10 kpps and phaseduration of 25 μs.

FIG. 3 shows various functional blocks in a signal processingarrangement for a hearing implant according to a prior art arrangement.

FIG. 4 shows various logical steps in developing electrode stimulationsignals according to an embodiment of the present invention.

FIG. 5 shows various functional blocks in a signal processingarrangement for a hearing implant according to an embodiment of thepresent invention.

FIG. 6 shows an example of a short time period of an audio speech signalfrom a microphone.

FIG. 7 shows an acoustic microphone signal decomposed by band-passfiltering by a bank of filters into a set of band pass signals.

FIG. 8 shows a broadband waveform signal for the syllable “bet”.

FIG. 9 shows a band pass filtered signal and envelope signal for thewaveform of FIG. 8.

FIG. 10 shows an envelope signal and smoothed envelope signal for thewaveform in FIG. 9.

FIG. 11 shows a section of the waveform signal from FIG. 9 with risingslope of the envelope.

FIG. 12 shows a section of the envelope signal and raised envelopesignal from FIG. 11.

FIG. 13 shows an example of a processed band pass signal and theresulting CSSS pulse sequence for a vowel with added Gaussian noise anda SNR=10 dB.

FIG. 14 shows the same signal as in FIG. 13, with SNR=5 dB.

FIG. 15 shows the same signal as in FIG. 13, with SNR=0 dB.

FIG. 16 shows an example of a processed band pass signal and SNR-adaptedpulse time intervals according to an embodiment of the presentinvention.

DETAILED DESCRIPTION

Parameters of a given cochlear implant stimulation coding strategy mightnot be optimal for all listening conditions. For example, in noisyconditions some stimulation coding strategies might perform better thanothers since temporal fine structure typically is more affected by noisethan is the band pass signal envelope. It would be beneficial to switchfrom one stimulation coding strategy to another, depending on listeningconditions. Depending on all the specific circumstances, the switchingmay usefully be performed in small increments so that the transitionhappens in a smooth morphing from one stimulation coding strategy to theother. Alternatively, in other situations, it may be beneficial todirectly switch between different stimulation coding strategies withouta transition period of time. Either way, the input sound signal or theband pass signals are monitored and analyzed to estimate one or more keyfeatures that are present. Based on the key feature(s), the stimulationcoding strategy is automatically modified.

FIG. 5 shows various functional blocks in a signal processingarrangement for a cochlear implant and FIG. 4 is a flow chart showingvarious logical steps in producing electrode stimulation signals toelectrode contacts in an implanted cochlear implant array according toan embodiment of the present invention. A pseudo code example of such amethod can be set forth as:

Input Signal Preprocessing:

-   -   BandPassFilter (input_sound, band_pass_signals)        Envelope Extraction:    -   BandPassEnvelope (band_pass_signals, band_pass_envelopes)        Stimulation Timing Generation:    -   TimingGenerate (band_pass_signals, stim_timing)        Key Feature Monitor:    -   KeyMonitor (band_pass_signals, key_feature)        Stimulation Coding:    -   CodingSelect (key_feature, coding_strategy)        Pulse Generation:    -   PulseGenerate (band_pass_envelopes, stim_timing,        coding_strategy, out_pulses)        The details of such an arrangement are set forth in the        following discussion.

In the arrangement shown in FIG. 5, the input sound signal is producedby one or more sensing microphones, which may be omnidirectional and/ordirectional. Preprocessor Filter Bank 501 pre-processes this input soundsignal, step 401, with a bank of multiple parallel band pass filters,each of which is associated with a specific band of audio frequencies;for example, using a filter bank with 12 digital Butterworth band passfilters of 6th order, Infinite Impulse Response (IIR) type, so that theinput sound signal is filtered into some K band pass signals, U₁ toU_(K) where each signal corresponds to the band of frequencies for oneof the band pass filters. Each output of the Preprocessor Filter Bank501 can roughly be regarded as a sinusoid at the center frequency of theband pass filter which is modulated by an amplitude envelope signal.This is due to the quality factor (Q≈3) of the filters. In case of avoiced speech segment, the band pass envelope is approximately periodic,and the repetition rate is equal to the pitch frequency. Alternativelyand without limitation, the Preprocessor Filter Bank 501 may beimplemented based on use of a fast Fourier transform (FFT) or ashort-time Fourier transform (STFT). Based on the tonotopic organizationof the cochlea, each electrode contact in the scala tympani typically isassociated with a specific band pass channel of the Preprocessor FilterBank 501. The Preprocessor Filter Bank 501 also may perform otherinitial signal processing functions such as automatic gain control (AGC)and/or noise reduction.

FIG. 6 shows an example of a short time period of an input speech signalfrom a sensing microphone, and FIG. 7 shows the microphone signaldecomposed by band-pass filtering by a bank of filters. An example ofpseudocode for an infinite impulse response (IIR) filter bank based on adirect form II transposed structure is given by Fontaine et al., BrianHears: Online Auditory Processing Using Vectorization Over Channels,Frontiers in Neuroinformatics, 2011; incorporated herein by reference inits entirety:

for j = 0 to number of channels − 1 do  for s = 0 to number of samples −1 do   Y_(j)(s) = B_(0j) * X_(i) (s) + Z_(0j)   for i = 0 to order − 3do    Z_(ij) = B_(i+1j) * X_(j)(s) + Z_(i+1j) − A_(i+1j) * Y_(j)(s)  end for   Z_(order − 2,j) = B_(order − 1j) * X_(i)(s) −A_(order − 1j) * Y_(j)(s)  end for end for

The band pass signals U₁ to U_(K) (which can also be thought of aselectrode channels) are output to an Envelope Detector 502 and aStimulation Timing Module 503. The Envelope Detector 502 extractscharacteristic envelope signals outputs Y₁, . . . , Y_(K), step 402,that represent the channel-specific band pass envelopes. The envelopeextraction can be represented by Y_(k)=LP(|U_(k)|), where |.| denotesthe absolute value and LP(.) is a low-pass filter; for example, using 12rectifiers and 12 digital Butterworth low pass filters of 2nd order,IIR-type. A properly selected low-pass filter can advantageously smooththe extracted envelope to remove undesirable fluctuations.Alternatively, if the band pass signals U₁, . . . , U_(K) are generatedby orthogonal filters, the Envelope Detector 502 may extract the Hilbertenvelope. In some embodiments, the Envelope Detector 502 may also beconfigured to determine one or more useful features of the band passenvelope such as envelope slope (e.g., based on the first derivativeover time of the envelope), envelope peak (ascending slope/positivefirst derivative followed by descending slope/negative firstderivative), and/or envelope amplitude of the band pass envelope.

The Stimulation Timing Module 503 processes selected temporal finestructure features for one or more of the band pass signals U₁, . . . ,U_(K) (typically, for one or more of the most apical, lowest frequencychannels) such as negative-to-positive zero crossings to generatestimulation timing signals X₁, . . . , X_(K). In the followingdiscussion, the band pass signals U₁, . . . , U_(K) are assumed to bereal valued signals, so in the specific case of an analytic orthogonalfilter bank, the Stimulation Timing Module 503 considers only the realvalued part of U_(k). For the selected band pass signals, theStimulation Timing Module 303 generates stimulation timing signalsreflecting the temporal fine structure features, step 403. In someembodiments, the Stimulation Timing Module 303 may limit theinstantaneous band pass frequency f₀ to the upper and lower frequencyboundaries f_(L1) and f_(U1) of the respective filter band. For example,a given band pass signal may have a lower frequency boundary f_(L1) of500 Hz and an upper frequency boundaries of f_(U1)=750 Hz.

A Key Feature Monitor 506 monitors one or more key features present inthe input sound signal or the band pass signals, step 404. For example,the key feature monitored by the Key Feature Monitor 506 may be thesignal to noise ratio (SNR) of the input sound signal, the direct toreverberation ratio (DRR) of the input sound signal.

A Stimulation Coding Module 507 is coupled to the Key Feature Monitor506 and selects a stimulation coding strategy for one or more selectedband pass signals based on the key feature value, step 405. Thestimulation coding strategies include: (1) an envelope-based stimulationcoding strategy based on the band pass envelope signals (such as CIS orHD-CIS that uses stimulation pulses at a constant stimulation rate), and(2) an event-based stimulation coding strategy that transmits temporalfine structure information based on the stimulation timing signals (suchas FSP or FS4 that uses adaptive stimulation rates according to thetemporal fine structure information). It is assumed that event-basedstimulation coding strategies are optimal in relatively quiet listeningconditions, while envelope-based stimulation coding strategies arebetter in noisier conditions. The Stimulation Coding Module 507automatically switches between different selected stimulation codingstrategies as a function of changes in the key feature value monitoredby the Key Feature Monitor 506. The Stimulation Coding Module 507 may beconfigured to switch between different selected stimulation codingstrategies directly without a transition period of time, or it mayswitch between different selected stimulation coding strategies byadaptively changing the selected stimulation coding strategy over atransition period of time to become a different stimulation codingstrategy; for example, either after or while the key feature valuechanges from an initial value to a coding change value.

A Pulse Generator 504 uses the stimulation coding strategy selected bythe Stimulation Coding Module 507 to produce the electrode stimulationsignals for the electrode contacts in the Implant 505, step 406. Forexample, the Pulse Generator 504 may be configured to creates CSSSoutput timing request pulses at the start of the negative to positivezero crossings of each of one or more selected band pass signals andthen weight (amplitude modulates) the CSSS stimulation pulses with theband pass envelopes from the Envelope Detector 502.

The Pulse Generator 504 also will typically further adjust outputelectrode stimulation signals based on a non-linear mapping thatreflects patient-specific scaling from the fitting process. Similarly,variations in perceived loudness in event-based stimulation coding andenvelope-based stimulation coding can be handled by adjustingstimulation parameters such as pulse amplitude, pulse duration, pulseshape or shape of the CSSS sequences (e.g. with short interpulseintervals). And again this may be based on patient-specific fittingparameters that are adjusted during presentation of speech signals untilspeech is perceived most naturally. In addition, the MCL and THR valuesmay vary when switching from one specific stimulation coding strategy toanother, so the MCL and THR values of the patient-specific scalingfunction should also be adjusted (in addition to the CSSS sequence) topromote a loudness-balanced transition between the different stimulationcoding strategies.

Some studies in normal hearing subjects (e.g., Dietz, Mathias, et al.“Emphasis of spatial cues in the temporal fine structure during therising segments of amplitude-modulated sounds.” Proceedings of theNational Academy of Sciences 110.37 (2013): 15151-15156; incorporatedherein by reference in its entirety) suggest that the fine structureinformation may be most effective during the rising slope of the bandpass envelope. Thus, some embodiments may use the rising slope of theband pass envelope as the key feature, or some other characteristic ofthe envelope. If the key feature is envelope peak, then event-basedstimulation coding may be applied during an interval defined around orfollowing the peak, the length of which may be related to the length andthe amplitude of the peak. A specific such arrangement may further beconfigured to use event-based stimulation coding reflecting the finestructure information only when the key feature is above some minimumthreshold value—when the rising slope of the envelope is great enough.Otherwise, envelope-based stimulation coding may be used. The envelopeslope may be averaged or low-pass filtered over recent past values inorder to provide smooth values.

For example, FIG. 8 shows a broadband waveform signal for the syllable“bet”. FIG. 9 shows a band-pass filtered version of this signal with alower frequency boundary of 200 Hz and an upper frequency boundary of325 Hz together with the band-pass envelope. FIG. 10 shows the band-passenvelope and a smoothed version of this envelope derived by low-passfiltering with 100 Hz. In FIGS. 11 and 12, the stimulation pulses arecontrolled by this smoothed envelope: if the slope of the envelopeexceeds a certain threshold (here 0.001), then fine structurestimulation is applied using single pulses at zero-crossings of theband-pass signal. These pulses at zero-crossings are weighted with thecorresponding band-pass envelope. At times where the envelope slope doesnot exceed the minimum threshold, envelope-only stimulation may beapplied using a regular time-grid for stimulation pulses where thepulses are weighted with the corresponding envelope signal.

As stated above, in some specific embodiments, the Stimulation CodingModule 507 may be configured to automatically make a smooth transitionbetween different stimulation coding strategies over a transition periodof time based on the SNR of the input sound signal by adaptivelymodifying the length and the shape of the stimulation timing signals(e.g., channel-specific sampling sequences (CSSS)). For each stimulationtiming signal from the Stimulation Timing Module 503, the StimulationCoding Module 507 determines an event-specific length for the CSSS pulsesequence (“FL interval”). The Pulse Generator 504 shapes the CSSS pulsesequence to follow the amplitude of the band pass envelope from theEnvelope Detector 502, in effect, sampling the band pass envelope withthe CSSS sequence.

When the SNR signal from the Key Feature Monitor 506 is relatively high(quiet sound environment), the Stimulation Coding Module 507 adjusts theFL interval to be so short that a CSSS pulse sequence may consist of aslittle as a single pulse. As the SNR signal from the Key Feature Monitor506 decreases (the environment becomes noisier), the Stimulation CodingModule 507 increases the FL interval and adds more pulses to the CSSSsequence until at some point for a low SNR (high noise), the last pulseof the CSSS sequence is seamlessly followed by the first pulse of thenext CSSS sequence, resulting in a continuous (constant rate) samplingof the band pass envelopes from the Envelope Detector 502. If the lengthof the FL interval becomes larger than the time between two consecutivetrigger events (i.e., two zero crossings), the Stimulation Coding Module507 may terminate the existing CSSS sequence when the next trigger eventoccurs and the FL interval of the following trigger event overrules theprevious FL interval. Or the Stimulation Coding Module 507 may continuewith the CSSS pulse sequence initiated by the first trigger event andignore the subsequent trigger event so that the end of the existing FLinterval, a new FL interval is determined. Once the SNR signal from theKey Feature Monitor 506 increases again, the Stimulation Coding Module507 adaptively adjusts the FS interval to again become shorter than thetimes between the trigger events.

FIG. 13 shows an example of a processed band pass signal and theresulting CSSS pulse sequence for a vowel with added Gaussian noise anda SNR=10 dB. The band pass signal is the higher frequency full sine wavesignal in green, the Hilbert band pass envelope is the slower varyinghalf sine wave trace shown in blue, and the vertical black linesrepresent the applied CSSS sequences with a sequence length of one. FIG.14 shows the same signal as in FIG. 13, when the SNR signal decreasesdown to 5 dB (more noise) and the FL interval is increased so that theCSSS sequences contain three pulses each. FIG. 15 shows the same signalas in FIG. 13, with SNR=0 dB (noisier still) where the FL interval is solong that the CSSS sequence performs a continuous sampling of the bandpass envelope that is similar to the HD-CIS coding strategy.

In some embodiments, the Stimulation Coding Module 507 may be configuredto switch between different selected stimulation coding strategies as afunction of changes in the key feature value monitored by the KeyFeature Monitor 506 and further as a function of more features of theband pass envelope from the Envelope Detector 502; for example, theenvelope amplitude, envelope slope and/or envelope peak value. Thus,where the key feature value is SNR, for high SNR, the Stimulation CodingModule 507 may trigger a CSSS sequence at each zero-crossing of the bandpass signal, while for lower SNR values, the Stimulation Coding Module507 may trigger a CSSS sequence only for zero crossings with a certainthreshold minimum envelope value (e.g., from the Hilbert envelope). Thisenvelope threshold functionality may be advantageous in noisyenvironments (low SNR) to better distinguish between zero crossingevents just caused by noise and those actually caused by a speech ormusical signal. The threshold minimum envelope may be channel-specificin some embodiments, while in other embodiments, it is not. Of course,some other key feature may be used, for example, direct to reverberationratio (DRR).

In addition to or alternatively to adaptively varying the length of theCSSS interval, other specific embodiments may adaptively control othersignal variables. For example, in combination with the application of aCSSS pulse at a specific event (e.g. a zero crossing event), asubsequent time interval—fine structure FS-interval—may be determinedwithin which a pulse has to be applied. The length of this FS-intervalmay be determined by the value of the SNR signal at the time when thepulse has been applied: If the SNR is high, the FS-interval may bechosen to be long, while if the SNR is low, the FS-interval may bechosen to be short. To restrict the stimulation rate to a maximum valuethat reflects the refractory period of the auditory nerve fibers, ashortest possible FS-interval can be defined that corresponds to themaximum stimulation rate. There are several different specificpossibilities:

-   -   If another timing event occurs within the FS-interval determined        from the previous timing event, and the time between the two        timing events is greater than the refractory period, then a        pulse can be applied at the second timing event and a new        FS-interval is initiated that overrules the previous one.    -   If another timing event occurs within the FS-interval determined        from the previous timing event, but the time between the two        timing events is shorter than the refractory period, then a        pulse can be applied at the end of the refractory period and a        new FS-interval is initiated that overrules the previous one.    -   If no additional timing event appears before the end of the        current FS-interval and the refractory period is shorter than        the FS-interval, then another pulse can be applied (forced) at        the end of the FS-interval.    -   If no additional timing event appears before the end of the        current FS-interval interval, but the refractory period is        greater than the FS-interval, then another pulse can be applied        (forced) at the end of the refractory period.        In general, an embodiment may require that a subsequent pulse        (either caused by the occurrence of a timing event or by the end        of the FS-interval) can only be applied if a minimum period        (e.g. the refractory period) is over. In some applications, it        may be advantageous to have a shorter period than the refractory        period as the minimum period.

FIG. 16 shows an example of a processed band pass signal and SNR-adaptedpulse time intervals according to an embodiment of the presentinvention. In FIG. 9, it is assumed that five zero crossing timingevents E1-E5 (vertical solid lines) are detected, the SNR is decreasingover time t (i.e. from left to right), and the refractory period is asshown by the corresponding horizontal arrows across the bottom thefigure denoted as “refractory time”. The first CSSS pulse is, withoutlimitation, applied at the zero crossing event E1. Since the SNR ishigh, the corresponding FS-interval I_1 is relatively long. The nextzero crossing event E2 occurs before the end of the I_1 FS-interval, sothe next CSSS pulse is applied at E2, and the same situation with I_2and E3 although the SNR has meantime decreased so that I_2 is shorterthan I_1. However, at the event E3′ that occurs at the end of the I_3FS-interval, a CSSS pulse is forced because no further zero crossing hasoccurred within this FS-interval starting after zero crossing event E3(also the refractory time (after E3) is still shorter than I_3). Thenext zero crossing event occurs at E4, but that is still within therefractory period after the last applied pulse at E3′ so there is nopulse applied at E4 nor is any corresponding FS-interval determined.Similarly, at E5 no pulse is applied, and in addition, at E4′ the SNR isso low that the corresponding I_5 FS-interval is determined to beshorter than the refractory period so at E5′, no pulse is applied butinstead is delayed until E5″ corresponding to the end of the refractoryperiod after event E4′.

When the SNR later increases more and more (not shown in FIG. 16) theFS-intervals will again become longer and longer until a zero crossingevent will be detected after the end of the refractory time but beforethe end of an FS-interval. From that point on, the zero crossing eventswill again determine the pulse sequence and the coding strategy followsthe known event-based coding strategies until the SNR decreases again.The maximum stimulation rate can be set to be proportional to theinverse of the minimum possible interval (e.g. the refractory period) sothat the instantaneous stimulation rate (which equals 1/FS-interval)cannot exceed a given defined value; e.g. a typical rate as presentlyused for CIS or HD-CIS coding strategies. In general, the lower the SNR,the more the resulting sound coding sequence will be according to anenvelope-based coding strategy such as CIS or HD-CIS (constant samplingof each channel in a prescribed manner with the defined maximumstimulation rate). The higher the SNR, the more the resulting soundcoding sequence will be like according to a pure event-based codingstrategy such as FSP.

The modification of the CSSS sequences can also be done channel-wise,i.e. based on channel-specific SNR values. And while the foregoing wasdescribed with SNR being the parameter for subsequent adaptivemodifications, other specific signal parameters that characterize thequality of an existing hearing situation may be used as well; e.g. thedirect to reverberation ratio (DRR).

Both approaches—variation of CSSS lengths and determination of timeintervals within which no pulse is applied—yield similar overallresults: a smooth transition between event-based (variable rate) andenvelope-based (constant rate) coding strategies. Embodiments of thepresent invention adapt the sound coding strategy to changes in thesound environment with optimal settings for each environment. WithSNR-adjusted sampling, temporal fine structure is provided in situationswhere it is not disturbed, while the sound coding is morphed seamlesslyto a more noise-robust envelope coding for better sound perception innoisier environments.

The foregoing discussion is presented in terms of switching thestimulation coding strategy on one or more band pass channelsindependently of the stimulation coding strategies used on any otherband pass channel. But embodiments of the invention includemulti-channel variants which determine key features for multiple bandpass channels and coordinate switching their stimulation codingstrategies together.

The band pass channels selected for coding temporal fine structure areanalyzed with respect to the selected key feature to identify thedominant key features in all the analyzed channels. In n-of-mstimulation coding arrangements, the band pass channels are typicallydivided into simultaneous time frames and the channel with largestenvelope amplitudes in each time frame typically are selected forstimulation. Or envelope slope may serve as the key feature where thedominant channel(s) are those with the largest slope(s). Or the keyfeature may be envelope peaks where the dominant key features may be thepeaks with the largest difference between their amplitudes to thesurrounding intra-channel noise level and/or with the smallestfull-widths-half-maximum (or a similar measure to characterize the widthof the peak). Similarly, if SNR or direct or reverberation ratio (DRR)are the key features, then the band pass channels with the largestvalues will be considered to be the dominant channels.

For example, in a multi-channel arrangement where envelope slope is usedfor the key feature, to enhance the transmission of temporal finestructure, only event-based channels (typically the most apical, lowestfrequency channels) would be analyzed for the envelope slope, the firstderivative in time of the envelope. The non-event-based channels can bestimulated in a regular CIS-type fashion without n-of-m selection. Onlyevent-based channels with a positive first derivative in time of theirband-pass envelope would be selected, and from these channels, a subsetof band pass channels with the largest envelope amplitude could bechosen for stimulation.

Assume a stimulation frame could comprise sequential ordered finestructure and envelope channels: CH1, CH2, CH3, CH4, CH5, CH6, wherechannels CH1, CH2 and CH3 are event-based fine structure channels andCH4, CH5 and CH6 are envelope-based channels. For the correspondingslopes of the envelopes SL1, SL2 and SL3, if SL2>0 and SL2>SL1 andSL2>SL3, then SL2 is the dominant key feature and the resultingstimulation frame with one selected event-based channel is CH2, CH4,CH5, CH6. If all the envelope slopes are negative for the event-basedchannels (no dominant key feature), then no event-based channel isselected and the resulting frame consists only of the envelope-basedchannels: CH4, CH5, CH6. With two selected event-based channels andSL2>0 and SL1>0 and SL2>SL3 and SL1>SL3, then SL1 and SL2 are thedominant key features and the resulting stimulation frame is: CH1, CH2,CH4, CH5, CH6. In all these examples, on the selected event-basedchannel(s), the stimulation signal is then developed and applied asdescribed above.

A multi-channel arrangement which determines key features for multipleband pass channels and coordinates switching their stimulation codingstrategies is not necessarily specific to an n-of-m strategy. Instead, asingle dominant key feature can be selected and event-based stimulationcoding can be applied on that channel and its neighbouring channels(provided they are event-based coding channels). So if envelope slope isthe key feature, then the channel with a maximum envelope slope isstimulated as described above, together in parallel with the adjacentneighbouring channels. The amplitudes of each of these correspondingstimulation pulses can be derived from the respective band pass envelopesignals similar to the CFS-based coding strategy described in U.S.Patent Publication 2009/0254150 (which is incorporated herein byreference in its entirety).

Embodiments of the invention may be implemented in part in anyconventional computer programming language such as VHDL, SystemC,Verilog, ASM, etc. Alternative embodiments of the invention may beimplemented as pre-programmed hardware elements, other relatedcomponents, or as a combination of hardware and software components.

Embodiments can be implemented in part as a computer program product foruse with a computer system. Such implementation may include a series ofcomputer instructions fixed either on a tangible medium, such as acomputer readable medium (e.g., a diskette, CD-ROM, ROM, or fixed disk)or transmittable to a computer system, via a modem or other interfacedevice, such as a communications adapter connected to a network over amedium. The medium may be either a tangible medium (e.g., optical oranalog communications lines) or a medium implemented with wirelesstechniques (e.g., microwave, infrared or other transmission techniques).The series of computer instructions embodies all or part of thefunctionality previously described herein with respect to the system.Those skilled in the art should appreciate that such computerinstructions can be written in a number of programming languages for usewith many computer architectures or operating systems. Furthermore, suchinstructions may be stored in any memory device, such as semiconductor,magnetic, optical or other memory devices, and may be transmitted usingany communications technology, such as optical, infrared, microwave, orother transmission technologies. It is expected that such a computerprogram product may be distributed as a removable medium withaccompanying printed or electronic documentation (e.g., shrink wrappedsoftware), preloaded with a computer system (e.g., on system ROM orfixed disk), or distributed from a server or electronic bulletin boardover the network (e.g., the Internet or World Wide Web). Of course, someembodiments of the invention may be implemented as a combination of bothsoftware (e.g., a computer program product) and hardware. Still otherembodiments of the invention are implemented as entirely hardware, orentirely software (e.g., a computer program product).

Although various exemplary embodiments of the invention have beendisclosed, it should be apparent to those skilled in the art thatvarious changes and modifications can be made which will achieve some ofthe advantages of the invention without departing from the true scope ofthe invention.

What is claimed is:
 1. A computer based method implemented using at least one hardware implemented processor for generating electrode stimulation signals to electrode contacts in an implanted cochlear implant electrode array, the method comprising: using the at least one hardware implemented processor to perform the steps of: processing an input sound signal to generate a plurality of band pass signals, wherein each band pass signal represents a band pass channel defined by an associated range of audio frequencies, and wherein each band pass signal has a characteristic amplitude and temporal fine structure features; extracting band pass envelope signals reflecting time varying amplitude of the band pass signals; generating stimulation timing signals reflecting the temporal fine structure features of the band pass signals; monitoring a key feature value present in the input sound signal or the band pass signals; selecting from the plurality of band pass signals a subset of one or more selected band pass signals; for the one or more selected band pass signals and for a plurality of stimulation coding strategies, selecting a stimulation coding strategy based on the key feature value, wherein the plurality of stimulation coding strategies include: i. an envelope-based stimulation coding strategy based on the band pass envelope signals, and ii. an event-based stimulation coding strategy based on the stimulation timing signals; using the selected stimulation coding strategy to produce the electrode stimulation signals; delivering the electrode stimulation signals to the electrode contacts in the implanted cochlear implant electrode array; and switching between different selected stimulation coding strategies as a function of changes in the key feature value.
 2. The method according to claim 1, wherein switching between different selected stimulation coding strategies occurs directly without a transition period of time.
 3. The method according to claim 1, wherein switching between different selected stimulation coding strategies includes adaptively changing the selected stimulation coding strategy over a transition period of time to become a different stimulation coding strategy.
 4. The method according to claim 3, wherein the adaptively changing occurs after the key feature value changes from an initial value to a coding change value.
 5. The method according to claim 3, wherein the adaptively changing occurs while the key feature value changes from an initial value to a coding change value.
 6. The method according to claim 1, wherein one of the stimulation coding strategies uses adaptive stimulation pulse rates and another stimulation coding strategy uses constant stimulation rates.
 7. The method according to claim 1, wherein the key feature value is a signal to noise ratio (SNR) of the input sound signal.
 8. The method according to claim 1, wherein the key feature value is a direct to reverberation ratio (DRR) of the input sound signal.
 9. The method according to claim 1, wherein the key feature value is envelope slope of the band pass envelope signals.
 10. The method according to claim 1, wherein the key feature value is envelope peak of the band pass envelope signals.
 11. The method according to claim 1, wherein the key feature value is envelope amplitude of the band pass envelope signals.
 12. The method according to claim 1, wherein the one or more selected band pass signals is a single band pass signal.
 13. The method according to claim 1, wherein the one or more selected band pass signals is a plurality of selected band pass signals.
 14. The method according to claim 13, wherein an n of m strategy is used with the plurality of band pass signals.
 15. The method according to claim 13, wherein the plurality of selected band pass signals comprises a dominant channel having a largest key feature value and a plurality of adjacent channels. 